Skip to content

Introduce retries to databricks psql command #3492

Merged
anton-107 merged 13 commits intomainfrom
anton-107/psql-retries
Aug 29, 2025
Merged

Introduce retries to databricks psql command #3492
anton-107 merged 13 commits intomainfrom
anton-107/psql-retries

Conversation

@anton-107
Copy link
Copy Markdown
Contributor

@anton-107 anton-107 commented Aug 26, 2025 •
edited
Loading

Why

Sometimes the connection to the database instance might fail, which can be remediated with a retry. This change makes the command to retry the connection 3 times by default.

To disable the retries users can set a max-retries value to zero:

$ databricks psql my-instance --max-retries 0

Tests

Added a new acceptance test, modified and existing acceptance test; ran manual tests on Mac and Windows

@anton-107 anton-107 marked this pull request as ready for review August 26, 2025 14:18
@eng-dev-ecosystem-bot
Copy link
Copy Markdown
Collaborator

eng-dev-ecosystem-bot commented Aug 26, 2025 •
edited
Loading

Run: 17299828292

Env ✅​pass 🔄​flaky 🙈​skip
🔄​ aws linux 299 9 504
🔄​ aws windows 303 6 503
✅​ aws-ucws linux 421 401
✅​ aws-ucws windows 422 400
🔄​ azure linux 302 6 503
🔄​ azure windows 305 4 502
🔄​ azure-ucws linux 413 8 400
✅​ azure-ucws windows 422 399
🔄​ gcp linux 298 9 505
🔄​ gcp windows 303 5 504
36 failing tests:
Test Name aws linux aws windows azure linux azure windows azure-ucws linux gcp linux gcp windows
TestAccept ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass ✅​pass
TestAccept/bundle/deploy/dashboard/simple ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky
TestAccept/bundle/deploy/pipeline/allow-duplicate-names ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass
TestAccept/bundle/deploy/pipeline/allow-duplicate-names/DATABRICKS_CLI_DEPLOYMENT=terraform ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass
TestAccept/bundle/deployment/bind/job/job-spark-python-task ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky
TestAccept/bundle/destroy/jobs-and-pipeline ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass
TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_CLI_DEPLOYMENT=terraform ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass
TestAccept/bundle/resources/pipelines/update ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass 🔄​flaky ✅​pass
TestAccept/bundle/resources/pipelines/update/DATABRICKS_CLI_DEPLOYMENT=direct-exp 🔄​flaky 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky
TestAccept/bundle/resources/pipelines/update/DATABRICKS_CLI_DEPLOYMENT=terraform 🔄​flaky 🔄​flaky 🔄​flaky ✅​pass ✅​pass 🔄​flaky 🔄​flaky
TestAccept/bundle/templates/default-python/combinations/classic 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=yes/PY=yes 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=yes/PY=yes ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass
TestAccept/bundle/templates/default-python/integration_classic ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/UV_PYTHON=3.10 ✅​pass 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/UV_PYTHON=3.11 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/UV_PYTHON=3.9 ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.10 ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.11 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.12 🔄​flaky 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=terraform/UV_PYTHON=3.9 ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass
TestAccept/selftest/record_cloud/pipeline-crud 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass
TestAccept/selftest/record_cloud/pipeline-crud/DATABRICKS_CLI_DEPLOYMENT=direct-exp 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass
TestAccept/selftest/record_cloud/workspace-file-io ✅​pass 🔄​flaky ✅​pass 🔄​flaky ✅​pass ✅​pass ✅​pass
TestAccept/selftest/record_cloud/workspace-file-io/DATABRICKS_CLI_DEPLOYMENT=direct-exp ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass ✅​pass
TestAccept/selftest/record_cloud/workspace-file-io/DATABRICKS_CLI_DEPLOYMENT=terraform ✅​pass 🔄​flaky ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass
TestFsLsForNonexistingDir ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass
TestFsLsForNonexistingDir/uc-volumes 🙈​skip 🙈​skip 🙈​skip 🙈​skip 🔄​flaky 🙈​skip 🙈​skip
TestFsLsOnEmptyDir ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass
TestFsLsOnEmptyDir/uc-volumes 🙈​skip 🙈​skip 🙈​skip 🙈​skip 🔄​flaky 🙈​skip 🙈​skip
TestFsLsOnFile ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass
TestFsLsOnFile/uc-volumes 🙈​skip 🙈​skip 🙈​skip 🙈​skip 🔄​flaky 🙈​skip 🙈​skip
TestFsLsWithAbsolutePaths ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass
TestFsLsWithAbsolutePaths/uc-volumes 🙈​skip 🙈​skip 🙈​skip 🙈​skip 🔄​flaky 🙈​skip 🙈​skip
TestGenerateFromExistingPipelineAndDeploy ✅​pass ✅​pass ✅​pass 🔄​flaky ✅​pass ✅​pass ✅​pass
TestLockUnlockWithoutAllowsLockFileNotExist ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass ✅​pass 🔄​flaky

denik
denik reviewed Aug 26, 2025
View reviewed changes
@anton-107 anton-107 force-pushed the anton-107/psql-retries branch from 0879a63 to 7dbc8ba Compare August 27, 2025 09:44
@anton-107 anton-107 requested a review from denik August 27, 2025 09:44
denik
denik reviewed Aug 27, 2025
View reviewed changes
@anton-107 anton-107 requested a review from denik August 27, 2025 11:55
@anton-107 anton-107 added this pull request to the merge queue Aug 29, 2025
Merged via the queue into main with commit b2c1182 Aug 29, 2025
13 checks passed
@anton-107 anton-107 deleted the anton-107/psql-retries branch August 29, 2025 11:20
deco-sdk-tagging bot added a commit that referenced this pull request Sep 3, 2025
@deco-sdk-tagging
[Release] Release v0.267.0
e6f94a3 
## Release v0.267.0

### CLI
* Introduce retries to `databricks psql` command ([#3492](#3492))
* Add rule files for coding agents working on the CLI code base ([#3245](#3245))

### Dependency updates
* Upgrade TF provider to 1.88.0 ([#3529](#3529))
* Upgrade Go SDK to 0.82.0

### Bundles
* Update default-python template to make DB Connect work out of the box for unit tests, using uv to install dependencies ([#3254](#3254))
* Add support for `TaskRetryMode` for continuous jobs ([#3529](#3529))
* Add support for specifying database instance as an application resource ([#3529](#3529))
* Allow referencing job libraries outside bundle root without the need to specify sync root ([#2842](#2842))
* Add top level `run_as` support for Lakeflow Declarative Pipelines ([#3307](#3307))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@denik denik denik approved these changes

@pietern pietern Awaiting requested review from pietern pietern is a code owner

@andrewnester andrewnester Awaiting requested review from andrewnester andrewnester is a code owner

@shreyas-goenka shreyas-goenka Awaiting requested review from shreyas-goenka shreyas-goenka is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@anton-107 @eng-dev-ecosystem-bot @denik